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(54) Barcode system 

(57) A barcode system (10) generates and decodes 
a barcode with visual significance. The barcode system 
(10) Includes an encoding module (14) for receiving a 
message (18) and a logo (16) and generating a visuaiiy 
significant barcode (20) based on these inputs. A de- 



coding module (24) is also provided for receiving an ac- 
quired version (48) of a visually significant barcode that 
may have been degraded in a typical document han- 
dling channel (e.g., through the printing, scanning, or 
copying process) and for recovering the message (60) 
specified by the visually significant barcode. 
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Description 

[0001] The present invention relates generally to bar 
codes and more specificaiiy to a method and apparatus 
for generating and decoding a visually significant bar- 
code. 

[0002] Barcodes are information carrying graphical 
patterns designed for easy and reilabie automatic re- 
trieval. The most common barcodes are known as one- 
dimepsionai barcodes. These graphical patterns vary in 
a single dimension (e.g. the horizontal dimension), and 
are constant in the other dimension (e.g., the vertical 
dimension). One-dimensional barcodes are employed 
in iow information content appiications like product in- 
dex registry (e.g. automatic price tagging and inventory 
management), or serial number registry {e.g. test-tube 
tagging in automated medical tests). Common exam- 
pies of one-dimensional barcodes are those barcodes 
that are affixed or printed on the packages of items pur- 
chased at a supermarket or department store. These 
barcodes typically can only encode limited information, 
such as the price of the item and the manufacturer. The 
items having the barcodes are scanned at a checkout 
counter to facilitate the tallying up of a total receipt. 
[0003] In order to convey more information on the 
same surface area, two-dimensional fea/rodes were de- 
veloped. Two-dimensional barcodes involve intricate 
patterns that vary in both the horizontal and the vertical 
dimensions. Two-dimensional barcodes are used in ap- 
plications that require more infonnation contents. For 
example, two-dimensional barcodes can be used to en- 
code mail! addresses for automated mail reading and 
distribution systems. Mail carrier companies can use the 
two-dimensional bar code on shipping packages to en- 
code shipper information , recipient information, tracking 
information, etc. in another example. t»vo-dimensional 
barcodes can be used to encode the compressed con- 
tent of a printed page to avoid the need for optica! char- 
acter recognition at the receiving end. 
[0004] U.S. Pat. Nos. 5,060,980, 5,486,686, and 
5,459,307 illustrate an exemplary 2D barcode system. 
This system utilizes short black bars that have a forward 
orientation or a backward orientation (e.g., bars that are 
oriented at either 45% or 135% with respect to a refer- 
ence) to render the barcode. The two possible orienta- 
tions of the bar aiiow information (e.g.. 1 or 0) to be en- 
coded therein. This system offers a limited improvement 
over the traditional 2D barcode in that the appearance 
of these short black bars while stilt distracting to the hu- 
man visual system is iess so than the traditional 2D bar- 
code. Consequently, it is asserted by these patents that 
a barcode utilizing these bars can be placed in the back- 
ground of a document without excessively distracting a 
person reading such a document. 
[0005] Unfortunately, the unsightly appearance of 2D 
bar codes reiegates these bar codes to impersonal com- 
merciai and industrial setting regardless of whether the 
barcode less distracting and can be incorporated into 



the background of a document. For example, in many 
consumer appiications, the current visuaiiy meaning- 
less barcode patterns are prohibitive, since users are 
more likely to decline the benefits of the application rath- 

s er than to irtciude a "barcode" on their letterhead. 
[0006] Another challenge in the design and use of 2D 
barcodes is the ability to produce barcodes by using of- 
fice equipment (e.g., printers) that can be subsequently 
read back also using office equipment (e.g.. scanners), 

io A problem is printer and scanner distortions that can 
vary across different types and manufacturers of the of- 
fice equipment. 

[0007] The present invention seeks to provide an im- 
proved barcode system. 
is [0008] According to an aspect of the present inven- 
tion, there is provided a visually significant barcode sys- 
tem as specified in claim 1 . 

[0009] According to another aspect of the present in- 
vention, there is provided a method of generating a vis- 

20 uaiiy significant barcode as specified in claim 8, 

[001 0] According to another aspect of the present in- 
vention, there is provided a method of decoding a visu- 
aiiy significant barcode as specified in claim 9. 
[001 1] The preferred embodiment can provide a bar- 

25 code which can be reiiabiy generated and scanned re- 
gardless of the type of machine and the manufacturer 
of the machine. The barcode can be robust to photocop- 
ying (i.e., the barcode can be scanned and information 
embedded therein decoded from a copy of the originally 

30 rendered barcode) and common office document deg- 
radation. 

[0012] The preferred embodiments can aiso provide 
a visually significant 2D bar code that is robust to dis- 
tortions in the paper path and that overcomes the dis- 
35 advantages set forth previously. 

[0013] The preferred barcode pattern with visual sig- 
nificance can be printed and read by standard office 
equipment. 

[0014] The preferred system includes an encoding 
4Q module for receiving a message and a logo and gener- 
ating a visually significant barcode based on these in- 
puts. A decoding rnoduie is also provided for receiving 
an acquired version of a visuaiiy significant barcode that 
may have been degraded in a typical document han- 
45 diing channel (e.g., through the printing, scanning, or 
copying process) and for recovering the message spec- 
ified by the visually significant barcode. 
[001 5] An embodiment of the present invention is de- 
scribed below, by way of example oniy, with reference 
50 to the accompanying drawings, in which: 

FIG. 1 is a biock diagram illustrating an office envi- 
ronment in which one embodiment of the visuaiiy 
significant bar code system that does not accept us- 
55 er input can be implemented. 

FIG. 2 is a biock diagram illustrating an office envi- 
ronment in which an alternative embodiment of the 
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visually significant bar code system that accepts us- 
er input can be implemented. 

FIG. 3 illustrates a fogo of a picture without any user 
input feature and a logo of a picture having a user 
input feature, 

FIG. 4A is a biock diagram illustrating an office ma- 
chine in which the visually significant bar code sys- 
tems of FiG. 1 or FIG. 2 can be implemented. FiG. 
4B is a block diagram illustrating a computer-read- 
able medium in which the visually significant bar 
code systems of FIG. 1 or FIG. 2 can be implement- 
ed. 

FIG. 5 is a biock diagram illustrating in greater detail 
the encoding modules and the decoding modules 
of FiG. 1 or FIG. 2. 

FIG. 8 illustrates a method of encoding a message 
into a visuaiiy significant barcode according to one 
embodiment of the present invention. 

FIG. 7A illustrates an exemplary implementation of 
the encoding step of FIG. 6 in accordance with one 
embodiment of the present invention. 

FIG. 78 illustrates pattern matrices according to 
one embodiment of the present invention that can 
be used in the encoding scheme of FiG. 7A. 

FIG. 8 illustrates a method of decoding a visuaiiy 
significant bar code according to one embodiment 
of the present invention. 

FIG. 9 illustrates an exemplary logo image and a 
corresponding visually significant barcode. 

FIG. 10 illustrates a zigzag scan that can be used 
in the decoding method of FiG. 6 to locate the bar- 
code zone. 

Visually Significant Bar Code System 10 

[0016] FIG. 1 is a block diagram illustrating an office 
environment in which one embodiment of the visuaiiy 
significant bar code system (VSBCS) 10 that does not 
accept user input can be implemented. The visually sig- 
nificant bar code system {VSBCS) 10 includes an en- 
coding module 14 for generating a visually significant 
bar code (VSBC) 20 by utilizing a rendering technique 
and a decod ing module 24 for decod ing an acquired ver- 
sion 48 of a hard copy (e.g., a first hard copy 33. a copy 
54 of the first hard copy 38, or a second hard copy 56) 
that includes a visually significant bar code {VSBC) and 
for recovering a message 60 that is specified in the ac- 
quired version 48. The VSBC 20 can be rendered by 
utilizing rendering techniques that are well-known to 



those of ordinary skill in the art. In one embodiment, the 
VSBC 20 is rendered by utilizing a halftone algorithm to 
generate a halftoned VSBC 20. 
[001 7] The system 1 0 can communicate with a driving 
s program 15 for providing the logo 16 and the message 
IS and aiso communicate with a driven program 70 for 
utilizing the recovered message 48 for a particular ap- 
plication. 

[0018] The encoding module 14 includes a first input 

w for receiving a iogo 16, a second input for receiving a 
message 18, and an output for generating the visuaiiy 
significant bar code (VSBC) 20. The message 18 can 
be any information that a system designer specifies. 
[0019] Logo 16 can be any graphical pattern that has 

75 visual significance, which is in sharp contrast to prior 2D 
barcodes that do not make sense as graphical entities. 
For example, logo 16 can include, but is not limited to, 
an actual logo, graphics, a picture, text, a text box, an 
image, or any other pattern that has visual significance. 

20 Exemplary logos can include a company logo, an appli- 
cation iogo, or action logo. Exemplary graphics can be 
frames, button marks, or background. An example of 
text is a text box having the words "This box contains 
important data!" The VSBC can be a bar code pattern 

?s that is embedded in the graphical design of text, pic- 
tures, images, borders, or the background of a docu- 
ment. 

[0020] The VSBC pattern may be, but is not limited 
to, a binary image (e.g., a bfack and white dot pattern), 
30 a mufti-level image {e.g., a gray-level image), or a mul- 
tilevel coior image that is generated by encoding module 
14, The encoding module 14 is described in greater de- 
tail hereinafter with reference to FIG, 5, 
[0C21] The office environment typically includes office 
35 equipment, such as a persona! computer (PC) (not 
shown), a printer 30 having a print engine 34, a scanner 
40 having a scan engine 44, a facsimile machine (not 
shown), and a copier 50 for making copies of documents 
(e.g.. copy 54). The print engine 34 receives images {e. 
4Q g., haiftoned images) from sources, such as the encode 
module 14, and renders these images to create hard 
copies, such as hard copy 38 with a first VSBC (L1 ). The 
scan engine 44 receives a hard copy (e.g., a hard copy 
38 or 54 with a first VSBC (L1 ) or a second hard copy 
45 56 with a second VSBC {L2)J, generates an acquired 
version 48 (e.g. a scanned version) of the received hard 
copy, and provides the acquired version 48 to applica- 
tions, such as decoding moduie 24. It is noted that the 
scanner 40 can be replaced with a digital camera for 
so acquiring a version of the visually significant barcode. 
Furthermore, the scan engine 44 can be any acquisition 
engine for capturing the VSBC 3nd providing an ac- 
quired version {e.g.. a digital representation) of the VS- 
BC. The operation and constraction of these office ma- 
ss chines and their respective components are generally 
wet! known by those of ordinary skill in the art and are 
not described herein. 

[0022] lrtatypicaldocument-handiingpath,theVSBC 
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along with the remainder of the document is provided to 
a printer, such as printer 30. The printer 30 then utilizes 
the print engine 34 to render a first hard: copy 38 that 
includes a first VSBC (L1 }. The hard copy 38 may then 
be provided directly to a scanner, such as scanner 44, 
to generate a scanned version 48. Alternatively, the hard 
copy 38 can first be copied once or several times utiiiz- 
ing a copier, such as the copier 50. Thereafter, the cop- 
ied version (e.g., copy 54) can be provided to the scan- 
ner 44 to generate the scanned version 48. It is noted 
that a second hard copy (e.g. , hard copy 58) can be gen- 
erated (e.g., printed or copied) by utilizing a printer (not 
shown) different from printer 30 that is part of a separate 
office. Aiso, it is noted that the iogo IS can vary from 
application to application (e.g., first VSBC (LI) of sec- 
ond VSBC (L2» and can be specified by a system de- 
signer 

[0023] The decoding module 24 receives the acquired 
version 48 and recovers the message 60 encoded in the 
acquired version 48. Fof example, if the acquired ver- 
sion corresponds to the first hard copy 38 or to the copy 
54, then the message in the first VSBC (L1 ) is recovered 
as the message 60. Aiternativeiy, if the acquired version 
corresponds to the second hard copy 56 or photocopies 
thereof, then the message in the second VSBC (12) is 
recovered as the message 60. The message 60 is then 
provided to a driven application 70 that performs one or 
more pre-determined functions or tasks, examples of 
which are described in greater detail hereinafter with ref- 
erence to FIG. 3. 

[0024] FIG. 2 is a block diagram illustrating an office 
environment in which an alternative embodiment of the 
visually significant barcode system tOAthat accepts us- 
er input can be implemented. This visually significant 
bar code system 10B is simii3r to the system 10A illus- 
trated in F!G. 1, except that this system 10A is config- 
ured to accept user input. A user can input information 
to the first hard copy 38, the copy 54, or the second hard 
copy 56 in the form of marks made by using a 'writing 
utensil (e.g., a pencil or pen) as will be explained in 
greater detail hereinafter with reference to FSG. 3. The 
system 10A includes a user input decoding module 26 
for decoding the user input, such as (1 ) user input 64 
that may be provided to the hard copy 38; or (2) user 
input 65 that may be provided to the copy 54 or the sec- 
ond hard copy 56. It is noted that the user input decoding 
module 26 may be integrated with the decoding module 
24 or be separate from the decoding module 24 as 
shown. 

[0025] In this embodiment, the decoded user input 66 
and the message 60 specified by the acquired image 48 
are provided to a decision module 68 that generates one 
or more signals based on the user input 66 and the mes- 
sage 60. These signals can specify or select from one 
of several options provided by the message. These sig- 
nals are then provided to one or more driven programs 
or applications 70 that perform pre-determined tasks or 
functions, launch other applications, or control other ap- 



S 

plications to perform the determined tasks. Examples of 
such task are described hereinafter with reference to 
FIG. 3. 

[0026] It is noted that the driving application 15 and 

s the driven application 70 can be the same appiication 
or different applications. Furthermore, the driving appli- 
cation 1 5 and the driven appiication 70 can be incorpo- 
rated into the visually significant bar code system (VS- 
BCS) 10 or be separate from the VSBC system 10. 

10 [0027] FIG. 3 illustrates a iogo 304 of a picture 304A 
without any user input feature and a logo 304B of a pic- 
ture having a user input feature. A document 300 (e.g., 
documents 38, 54, or 56) includes a non-iogo portion 
302 (e.g., text, Images, pictures etc.) and a iogo portion 

15 304 (e.g., first VSBC (L1 ) or second VSBC <L2)) that is 
can be a VSBC 20. It is noted that the VSBC 20 is pro- 
vided along with the non-iogo portion so that a printer 
(e.g., the printer 30) can render the entire document 
300. Since the rendering and decoding non-iogo por- 

2» tions of documents are well known by those of ordinary 
skill in the art, these operations are not described herein. 
[0028] The logo 304 can be a picture 304A without 
any user input feature or a picture 304B wi th a user input 
feature. In either case, the picture (304A or 304B) in- 

?s eludes an encoded message 308 that can be, but is not 
limited to, an internet address (e.g., a World Wide Web 
address), an electronic mail address, authentication in- 
formation (e.g.. a digital signature or the author or send- 
er of the document), a file name and file location, a fac- 

30 simife number for "fax-back" applications, or a combina- 
tion thereof. 

[0029] One embodiment provides a VSBC to increase 
the value of documents and to create "living docu- 
ments". Living documents are those documents that 

35 have more information (e.g.. encoded messages) than 
meets the eye or is apparent in viewing the documents. 
Living documents have encoded messages in the form 
of VSBCs that can be used to dynamicaiiy retrieve ad- 
ditional information (e.g.. updated versions of the doc- 

4Q ument). to reduce human intervention, to increase con- 
venience, to improve efficiency in the office, to increase 
security in the distribution of documents, to increase 
productivi ty and to facilitate interaction and communica- 
tion between people in the office. 

45 [0030] There are numerous applications that use one 
or more of the types of messages, described above, and 
encodes these types of messages into documents, it is 
to be appreciated that there are many other types of 
messages or information that may be encoded into doc- 

50 uments and numerous applications related thereto. The 
cornerstone of these applications is the VSBCS de- 
scribed herein. 

[0031] For example, one application is to use the VS- 
BC to store the file name and the fife location (e.g., path 
55 to access the document on a server, Web site, etc.). 
When a document having a VSBC that contains file and 
file location information is scanned into or otherwise ac- 
quired by a personal computer, a Most-Recent Docu- 
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menf .Application can utilize this information to access 
the most recent or updated electronic version of the doc- 
ument and Haunch the document by using a word 
processing application. 

[0032] As can be appreciated, the system can provide 
significant improvements over the prior art approach 
that would attempt to convert the scanned version into 
a text-edrtabie version via some type of OCR. The OCR 
approach is prone to many errors and mistakes in rec- 
ognition, tn addition, regardless how accurate the OCR 
utilized, only the version as shown on the physicai doc- 
ument can be accessed, if that document has been sub- 
sequently revised, there is no mechanism that would ai- 
iow for the retrieval of the most up-to-date version. This 
appiication retrieves the most-current electronic version 
even if the document has poor quality text that is inad- 
equate for recognition by prior art OCR methods be- 
cause the fife name and file path are encoded in the VS- 
BC. 

[0033] Similarly, the encoded name and location of 
the original: electronic version of the document, allows 
a copier to utilize the original document to be utilized as 
a copy source instead of having to rely on the document 
in-hand with the VSBC, which may have been severeiy 
degraded in the typical paper handling path. This is an 
exampie of a Copy-from-Electronic- Version Application. 
[0034] A security application can utilize authentication 
information encoded in the VSBC for authentication pur- 
poses or other secure communication functions. For ex- 
ample, a digital signature can be encoded into the VSBC 
so that the source of the document can be authenticat- 
ed. 

[0035] In another example, the facsimile number is 
encoded in the VSBC so th3t an Automatic Fax-Back 
Appiication can utiiize the encoded facsimile number to 
automatically dial the number when the document is fed 
into a facsimile machine without user intervention. Al- 
ternatively, an e-maii address may be encoded in the 
VSBC so that an Automatic Electronic-Mail Response 
.Appiication can utilize the encoded electronic mail ad- 
dress to automatically send more information (e.g., a re- 
sponse) via e-mail to the author of the document. 
[0038] The picture 304B with a user input feature can 
include one or more user input forms. For exampie. a 
Check Box Form 3 1 4 can be provided for a user to select 
from one of several options by simpiy checking the ap- 
propriate box. The text message box "Check Appropri- 
ate Box" 315 can be a visually significant barcode. In 
another example, a Circle Selection Form 318 can be 
provided for a user to select from one of several options 
by simply circling the appropriate selection. The text 
message box "Circle Desired Option" 319 can be a vis- 
uaiiy significant barcode. In yet another example, a Fill- 
in Form 324 can be provided for a user to fill in informa- 
tion requested by the form. The text message box "Fill 
in" 325 can be a visually significant barcode, 
[0037] Each of these forms request user input 330. 
such as a check mark, a circie, or text input. The user 



S 

input 330 can then be decoded by the user input decode 
moduie 26 to recover the user input. Thereafter, the de- 
cision moduie 68 utilizes the user input in conjunction 
with the recovered message to select one of several 

s available functions or tasks, and the driven application 
70 for performing the selected tasks can be launched. 
For exampie. the user input can be utilized to arbitrate 
between different messages encoded in the different 
visually significant text boxes. In the automatic e-mail 

10 response application, the user input and the different 
visually significant text boxes can be utilized to (1 } reply 
only to the sender or {2) to reply to all recipients including 
the sender. 

[0038] The VSBC system 10 or 1 0A can be embodied 
is on a media that is incorporated in office equipment (e. 
g., a personal computer (PC), printer, facsimile ma- 
chine, scanner, copier, all-in-one machine, etc.) or sep- 
arate from office equipment. When incorporated in office 
equipment, the media, having the VSBC system 10 or 
2» 1QA embodied therein, can be in the form of a memory 
(e.g., random access memory (RAM), read only mem- 
ory (ROM), etc.). When incorporated separate from the 
office equipment, the media, having VSBC system 10 
or 10A embodied therein, can be in the form of a com- 
?s puier-readable medium, such as a floppy disk, compact 
disc (CD), etc. 

[0039] FiG. 4A is a block diagram illustrating an office 
machine 400 in which the VSBC system 10 or 10A can 
be implemented. The office machine 400 includes a 

so memory 410 in which the VSBCS 10 or 10A of the 
present invention can reside. The office machine 400 
may be a persona! computer (PC). If this is the case, the 
PC can be coupled to a printer, scanner, an all-in-one 
machine or other peripheral machine. It Is noted that the 

35 VSBCS 1 0 or 1 0A may reside in the PC in the form of a 
driver program associated with one of these peripheral 
machines or can be stored in a memory that is located 
in the peripheral machines. 

[0040] Alternatively, the office machine can be an aii- 

4Q in-one office machine (e.g., an integrated scanner, print- 
er, and facsimile machine) or any rendering machine. 
[0041] FIG. 4B is a block diagram illustrating a com- 
puter-readable medium 420 in which the VSBC system 
10 or 10A of the present invention can be implemented. 

45 The computer-readable medium 420 may be a floppy 
disk 440, a compact disc (CD) 430, or other computer- 
readable medium, etc. As can be appreciated, the VS- 
BC system 10 or 1 0A can be implemented in a variety 
of other ways and in a variety of other components that 

50 are commonly found in an office environment. 

[0042] FIG. 5 is a block diagram illustrating in greater 
detaif the encoding moduie 14 and the decoding module 
24 of FIG. 1 and FiG. 2. The encoding moduie 14 in- 
cludes an encoder 500 for receiving a message 18 and 

55 generating an encoded message 504 based on the 
message 18. Encoding module 14 also includes a 
graphic modulation unit 510 for receiving the logo 16 
and the encoded message 504 and based thereon, gen- 
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crates the VSBC 20. The VSBC 20 can be a specific 
pattern. In the channel 514, the VSBC 20 can be ren- 
dered directly on a document or incorporated into a doc- 
ument and rendered thereon. 

[0043] Preferably, the visually significant two-dimen- 
sional barcodes are graphical patterns composed, usu- 
affy though not necessarily, of dots. The visually signif- 
icant baf codes are rendered, usuaiiy though not nec- 
essarily, using too-toned dots (e.g. black dots on a white 
background), and occupy, usually though not necessar- 
ily, a rectangular area. The visually significant bar codes 
can incorporate various registration and fiducial marks, 
which are generally well known to those of ordinary skill 
in the art, for enabling automated identification and ac- 
curate registration of the bar code, which may be read 
in arbitrary orientations. In addition, the visually signifi- 
cant bar code system can employ various error correct- 
ing codes, which are generally weii known to those of 
ordinary skill in the art, for reiiabie automated retrieval 
of the bar codes. 

[0044] The steps carried out by encoding module 14 
to process the message 18 and the logo 16 to generate 
the VSBC 20 are described in greater detail hereinafter 
with reference to FIG. 6. 

[0045] Channel 514 represents a typical office docu- 
ment-handling path for hard copies by standard office 
equipment. For exampie, a hard copy can be printed, 
photocopied, and scanned. It is understood that one or 
more of these standard office handiing steps can be car- 
ried out by muiti-fuectionai equipment. Since the bar- 
code pattern is subject to noise and office document 
degradation (e.g., foids, stains, marks, and staples) in 
the channel 514, a standard cleaning process to ad- 
dress the noise and degradation of barcode pattern can 
be employed. 

[0046] The steps carried out by the decoding module 
24 to recover the message 60 from the acquired image 
48 are described in greater detail hereinafter with refer- 
ence to FIG. 8. 

Generating the VSBC 20 

[0047] FiG. 6 illustrates a method of encoding a mes- 
sage 18 into a visually significant barcode 20 according 
to one embodiment of the present invention. In step 600, 
the message 18 and the logo 16 are received. The mes- 
sage 1 8 can also be compressed by util izing well known 
compression techniques in this step. For certain appli- 
cations, such as universal resource locators (URLs), a 
tailored or customized compression scheme may be 
preferable. 

[0048] In step 61 0, the compressed message is en- 
coded by utilizing an error correction code with an output 
alphabet L to generate a message encoded in a se- 
quence of Q symbols. This encoding step provides ro- 
bustness to errors in the channel 514. For exampie, a 
standard 16 -> 31 bit BCH code that corrects for three 
errors, where L = 4 (2 bits) can be used. 



[0049] In step 620, the logo 1 6, which can be an M x 
N pixel image, is partitioned in a plurality of image ma- 
trices. In one example, K is equal to two (2), and M and 
N (i.e., the size of the logo 16) are even values {e.g., N 
s = 80 and M = 40). The values of K. M and N can be 
modified and predetermined by a system designer to 
suit a particular application. 

[0050] In step 630, an image area corresponding to R 
logo matrices is used for pre-defined fiducial marks. For 

10 example, the four comers of the image can be reserved 
forftducia! marks, in each comer, an area of 4 x 4 pixels 
(2x2 matrices) can be designated for fiducial marks. 
The fiducial marks can be the whole area rendered 
white, except for an isolated location in each comer that 

is is set apart or separate from adjacent matrices, which 
is rendered black. The upper-left fiducial mark, for ex- 
ample, can be all white except for the upper-left pixel, 
which is rendered black, it is preferable that the pattern 
for the fiducial mark be as robust to channel noise as 

20 possible. Since dots may be blurred or move relative to 
each other due to noise in the channel 514, this pattern 
ensures that the black fiducial dots do not merge with 
neighboring dots, and instead stand out clearly on a 
white background. Various other fiducial patterns can be 

25 utilized depending on the particular types of noise in a 
particular channel of interest, 

[0051] In step 640, the remaining logo matrices (i.e., 
P = Mx Nl(K xK}-R logo matrices) are converted to K 
x /(binary barcode matrices by utilizing one of a prede- 

30 fined set of L distinct maps (e.g. , halftoning algorithms) 
based on a corresponding symbol in the coded mes- 
sage. In one example, there are 784 remaining image- 
matrices (based on N =80 and M =40) that can be or- 
dered in raster scan. The 784 image matrices can ac- 

35 commodate slightly more than 25 batches of 31 matri- 
ces (Q = 25 x 31 = 775). It is important to note that P 
should be greater than Q. Each batch codes two batch- 
es of 16 input bits: (1 ) one batch for the BCH MS bits, 
and (2) the other batch for the BCH LS bits. Thus, for 

40 the parameters specified above, the barcode can en- 
code 25 x 15 • 2 = 800 bits of information. 
[0052] FIG. 7A illustrates an exemplary implementa- 
tion of encoding step 640 of FIG. 6 in accordance with 
one embodiment of the present invention. The logo 16 

45 incfudes a plurality of pixels having one of two different 
gray levels: 25% and 75%. For example, pixels 704, 706 
have a 25% gray level, and pixel 708 has a 75% gray 
level. 

[0053] Referring to FIG. 7B, for those pixels in the logo 
so isthat have a 25% gray level, one of the four barcode 
matrices in the first row is used to render the based on 
the data to be encoded. For example, if the data to be 
encoded is "11", pixel 704 is rendered by utilizing the 
barcode matrix710. Similarly, if the data to be encoded 
55 )s "01", pixel 706 is rendered by utilizing the bar code 
matrix 730, Similarly, for those pixels in the logo 18 that 
have a 75% gray level, one of the four bar code matrices 
in the second row is used to render the based on the 
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data to be encoded. For example, if the data to be en- 
coded: is "00", pixel 708 is rendered by utilizing the bar 
code matrix 720. 

[0054] Although this implementation utilizes two gray 
levels, 25% and 75%, it is noted that the system can be 
extended to include multiple levels can be utilized to suit 
a particular application, in addition, it can be extended 
to pixels of different colors. 

[0055] Alternatively, the remaining image-matrices 
can be ordered with interleaving methods that are 
known in the art, to provide robustness to burst-type 
degradations expected from stains, marks, or system- 
atic printer/scanner distortions. The K x K binary bar- 
code matrices can then be sent to a printer for printing 
onto a hard copy, such as a piece of paper. 
[0056] There are many rendering methodologies from 
which one can choose the L distinct maps. For exampie, 
when halftoning algorithms are selected as the L distinct 
maps, there are many type of halftone methodologies 
one can choose from, in an exempiary implementation 
a flxed-halftone-pattern halftoning method is utilized. 
This method specifies that the image be a 2-tone image, 
if black is 0, and white is 1 , the bright tone b, and the 
dark tone d, are such that d= 1 - 6. Aiso, the L halftoning 
algorithms correspond to L distinct K x K pattern-matri- 
ces, where each pattern matrix contains b * K x K black 
dots on white background . However, it is noted that oth- 
er well-known halftoning methods, such as duster dith- 
ering, disperse dithering (e.g., blue noise), and error dif- 
fusion methods can be utilized. When an error diffusion 
algorithm is employed, one can select from many differ- 
ent methods to diffuse the error. Similarly, when a dis- 
perse dithering algorithm is utilized, one can seiect from 
many methods to define the dither matrices (also known 
as screens) of various sizes. 

[0057] Given a logo-matrix, and the selected pattern- 
matrix, the resulting barcode-matrix contains the pattern 
matrix values in the places corresponding to the bright 
pixels in the logo-matrix, and their complementary oth- 
erwise. 

[0058] In this embodiment, the dots are preferably 
rendered as square dots at 120 dots-per-inch (dpi). It is 
noted that larger dots are more robust to channel deg- 
radations, but smaller dots en3bie more information on 
the same area of the paper. The dots can be rendered 
at a dpi higher than 120dpi as iong as acceptable error 
resilience can be assured. 

[0059] It is noted that the size of the dots can be varied 
according to intensity. An exampie of varying the size of 
dots based on intensity is to make the black dots slightly 
larger than the white dots. It is aiso noted that the size 
of the dots can be varied according to neighborhood. An 
exampie of varying the size of dots based on neighbor- 
hood is to make minority coiofed dots larger. 
[0080] FIG. 9 frustrates an exempiary logo 15 and 
message 18 that are used to generate a corresponding 
VSBC 20 by the disclosed encoding method. 



Decoding the Acquired Barcode image 48 

[0061] The image 48 acquired by the scanner 40 and 
introduced to the decoding module 24 is a degraded ver- 
s sion of the origi nal bar code-image. These degradations 
are attributed to the channel 514, namely, the printing 
and scanning processes, and potential office type deg- 
radations, such as copying, stains, folds, staples, and 
marks. 

to [0062] FIG. 8 illustrates a method of decoding a visu- 
aiiy significant barcode according to one embodiment of 
the present invention. In step 800, the acquired image 
is feceived, fof example, from the scanner, in step 810, 
the bar code image is located in the recei ved image. For 

is exampie, the received image can be a whoie page of 
information, whereas the barcode image can be limited 
to a barcode zone, which may be a portion of the whole 
page, in one embodiment, the barcodes are located 
roughly at a same predetermined location in the ac- 

23 quired image and are surfounded by white pixels. This 
predetermined location is referred to herein as the bar- 
code-zone. Alternatively, the barcodes can be disposed 
in varying locations on a page, as background, or as a 
frame for the page. When barcodes are disposed in 

?s such a manner, a barcode locating procedure can be 
utilized to locate the barcode image on the page. 
[0063] In step 820, the fiducial marks are detected. 
The configuration of these fiducial marks indicates the 
type of global, deformation introduced by the channel. 

30 in one embodiment, the fiducial marks are detected by 
using a zigzag scan. 

[0064] FIG. 10 illustrates a zigzag scan 1000 that can 
be used in step 820 to detect the fiducial marks. The 
zigzag scan 1000 begins at the upper-left comer 1010. 

35 The first dark pixel 1020 of each scan (requires a thresh- 
old) is detected and considered to be a part of the cor- 
responding fiducial pixel. The first dark pixel 1020 is also 
used as an anchor pixel for that mark (red dot in Figure 
5). Next, a flood-fill algorithm, which is known in the art, 

40 locates aii the dark pixels 1030 connected to these an- 
chors. The centers 1040 of toe fiduciai marks are then 
computed as the average (center of mass) of the pixels 
of each mark. An advantage of this zigzag scan is that 
it enables a robust detection of toe anchor points even 

45 in the presence of significant rotations. This scan can 
be repeated at each comer of the barcode. 
[0065] In step 830, viewing transformation distortions 
(e.g.. translation, rotation, and afflne) are corrected. By 
setting the center of the coordinate system to be ten pix- 

50 eis above and ten pixels to the left of the center of the 
upper-ieft fiducial mark, translation problems can be 
eiirninated. The relative rotation of the center of the up- 
per-rightfiduda! mark is then determined, and the image 
is rotated back accordingly. 

55 [0066] Next, the skew factor, which is the horizontal 
translation of the centers of Sower marks relative to the 
horizontal translation of the centers of the upper mark, 
is determined, if a skew is detected, the image is cor- 
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rected for the detected skew. These transformation pro- 
cedures can be implemented with standard procedures 
known in the computer vision and image processing ar- 
eas. 

[0087] In one embodiment, bilinear interpolation is uti- 
lized although other interpolators can be used, it is noted 
that a singie sffine transformation can be utilized to cor- 
rect for both the rotation and the skew. 
[0088] The four fiducial marks enable correction of 
giobai transformations with up to eight degrees of free- 
dom, in this embodiment, there are four degrees of free- 
dom {two - translation, one - rotation, and one- skew). 
[0089] In step 840, the acquired barcode image is par- 
titioned into a plurality of sub-images that can be ar- 
ranged in a rectangular array, where each sub-image 
corresponds to a singie barcode matrix. Step 330 pro- 
vides a rectangular image. In one embodiment, the step 
of partitioning the acquired barcode image invoives the 
steps of measuring the image and slicing the image into 
rectangular sub-images. In our case M/K = 20, and N 
/K = 40 so each rectangular sub-image had the dimen- 
sions of 20 x 40. 

[0070] In step 850, each sub-image is compared 
against the set of L possible barcode-matrices (i.e., the 
outputs of the L distinct maps {e.g., halftoning algo- 
rithms) given the corresponding image). When the half- 
tone patterns of FIG. 7B are used, and the logo design 
is required to have a constant brightness in every logo- 
matrix, four vector products can be appiied to every sub- 
image. The four vectors are Gaussian profiles that are 
each centered at the center of a quadrant of the sub- 
image. For dark sub-images, the pattern of FIG. 7B cor- 
responds to the quadrant having the maximal value, and 
for bright sub-images, the pattern corresponds to the 
quadrant having the minima! value. This approach is al- 
so known as the matched filter approach in the image 
processing arts. 

[0071] In step 860, a best match is selected to repre- 
sent the sub-image in a sequence of P symbols over 
{1,2,... L). it is noted that any maximum-likelihood-type 
of detector or any other match estimator can be utilized 
to determine which of the L possible maps (e.g.. half- 
tones) is the most likely to have produced the corre- 
sponding sub-image. Preferably, the best match is per- 
formed on a group of sub-images. 
[0072] In step 870, the (possibly erroneous) se- 
quence that was originally coded with an error-correct- 
ing code is decoded, thereby eliminating the effect of 
possible errors due to the degradations of the channel 
The corresponding standard BCH error correction may 
be performed followed by the appropriate re-ordering of 
the bits. 

[0073] In step 880. the sequence is decompressed to 
give the original message. 

[0074] The present invention provides a barcode pat- 
tern with visuai significance, thereby increasing its ac- 
ceptance and use in the business environment and even 
the home environment. Whereas the prior art bar codes 



were limited to impersonal commercial and industrial 
applications, the barcode pattern with visuai signifi- 
cance changes the sentimental vaiue attached the use 
of barcodes in more personal contexts from unfavorable 

s to favorabie. 

[0075] In addition, different applications can easily in- 
corporate the visually significant barcode pattern and 
different variations thereof (e.g. , different logos or other 
graphics), thereby enabling a user to customize or per- 

10 sonalize the barcode pattern. 

[0076] Moreover, the system utilizes techniques, such 
as error correction and the use of fiducial marks, to make 
the visually significant barcode pattern robust to imper- 
fections and noise in the hardcopy-handling channel 

is For exampie, the barcode pattern is robust to consecu- 
tive photocopying by common office copiers {e.g., ana- 
log and digital copiers). Aiso, the barcode pattern is re- 
silient to common office document degradations, such 
as folds, stains, marks, and staples. 

20 [0077] Furthermore, the visually significant barcode 
pattern can be printed by and read by standard office 
equipment, such as printers, scanners, copiers, and 
multi-functional equipment. 

[0078] The disclosures in United States patent appfi- 
25 cation no. 09/579,070, from which this application 
claims priority, and in the abstract accompanying this 
application are incorporated herein by reference. 



Claims 

1 . A visually significant barcode system (1 Q) including: 

an encoding module (14) for recei ving a mes- 
sage (18) and a logo (16) and based thereon for 
generating a visually significant barcode (20) hav- 
ing the message encoded therein. 

2. A system as in claim 1 , including: 

a print engine (34) coupled to the encoding 
module for receiving the visuaiiy significant barcode 
and based thereon for rendering a hard copy of the 
visually significant barcode. 

3. A system as in claim 2, wherein the print engine uti- 
lizes a halftone algorithm to render the hard copy of 
the visually significant barcode. 

4. A system as in any preceding claim, including: 

a decoding module (24) for receiving an ac- 
quired version (48) of the visually significant bar- 
code and based thereon for recovering a message 
(SO) encoded therein. 

5. A system as in claim 4, wherein the encoding mod- 
ule and the decoding module are embodied on a 
medium. 

6. A system as in claim 5, wherein the media is incor- 
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porated in an office machine in the form of a mem- 
ory. 



A system as in claim 4, 5 or 6, including: 

an acquisition engine (44) coupled to the de- 5 
coding module for receiving a hard copy having a 
visually significant barcode and, based thereon, for 
generating the acquired version of the visually sig- 
nificant barcode. 

io 

A method^ for generating a visually significant bar- 
code comprising; 



receiving an M x N: pixel image (600); 

receiving a message having a piuraiity of fields fs 

{600}: 

partitioning the M x N pixel image into a plurality 
of K x K image matrices (620); and 
converting the K x K image matrices to K x K 
barcode matrices by utilizing one of a predeter- a? 
mined set of L distinct maps; wherein the se- 
lection of the particuiar map is based on a cor- 
responding field of the message (640). 

A method of decoding a visually significant barcode ?5 
comprising: 



receiving the barcode image (800); 
partitioning the barcode image into a piuraiity 
of sub-images (840); so 
comparing each-sub-image with a set of L pos- 
sible barcode matrices (850): and 
decoding a message based on a match estima- 
tion of each sub-image to each one of the L pos- 
sibie barcode matrices in a sequence of P sym- 35 
boisover{1,2,...L}{860). 
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RECEIVING A MESSAGE AND A LOGO (MAGE 






ENCODING THE MESSAGE BY UTILIZING AN ERROR 
CORRECTION COOE WITH AN OUTPUT ALPHABET L 
TO GENERATE A MESSAGE ENCODED IN A 
SEQUENCE OF Q SYMBOLS 






PARTITIONING A LOGO IMAGE, WHICH CAN BEAJtfX 
N PIXEL IMAGE, INTO A PLURALITY OF LOGO 
MATRICES 






DEFINING AN IMAGE AREA FOR PREDETERMINED 
FIDUCIAL MARKS 
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CONVERTING THE LOGO MATRICES (EXCEPT FOR 
THOSE LOGO MATRICES USED FOR FIDUCIAL MARKS) /-640 
TO KXK BINARY BARCODE MATRICES BY UTILIZING v 
ONE OF A PREDEFINED SET OF L DISTINCT MAPS 
8ASED ON A CORRESPONDING SYMBOL IN THE 
CODED MESSAGE 
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RECEIVING AN IMAGE (E.G. A SCANNED IMAGE) r 80G 
HAVING A BARCODE IMAGE 



LOCATING THE BARCODE IMAGE IN THE RECEIVED 
IMAGE 
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DETECTING FIDUCIAL MARKS IN THE BARCODE 
IMAGE 
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CORRECTING VIEWING DISTORTIONS 
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PARTITIONING THE BARCODE IMAGE INTO A ^840 
PLURALITY OF SUB-IMAGES 



COMPARING EACH SUB-IMAGE WITH A SET OF L 
POSSIBLE BARCODE MATRICES 
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REPRESENTING EACH SUB-IMAGE AS A BEST MATCH 
OF ONE OF THE I POSSIBLE BARCODE MATRICES IN 
A SEQUENCE OF P SYMBOLS OVER \\, 2. .. i) 
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